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DESCRIPTIQtf 

Synaptic Element: And Array 

1- Field Of The Invention 

The present invention relates to artificial neural 
networks and synapses. More particularly, the present 
invention relates to a single adaptable synapses and an 
5 array of adaptable synapses. 

2. The Prior Art 

Several schemes for using a matrix of electronic 
devices for neuron network applications have been 
proposed. To date, all such schemes involve using "weigh- 

10 ts" to control the amount of current injected into an 
electrical node "neuron". These weights were set by 
controlling the value of a resistor or the saturation 
current of a transistor. The limitation of any such 
scheme is that the value of any parameter of an electronic 

15 device in an integrated circuit is not well controlled. 
For example, the saturation currents of two MOS 
transistors of the same size can be different by a factor 
of 2 if these devices are operated in the sub-.threshold 
regime. The "training" mechanism that adjusts the weights 

20 must take these uncertainties into account by iterating 
and testing the outcome of the weight-adjustment process. 
It is therefore desireable that an adaptive mechanism be 
found whereby the matrix element adjusts itself to any 
uncertainty in device parameters, as part of the training 

25 process. 

Brief Desc ription Of The Invention 

In a first aspect of the present invention, an 
adaptable CMOS inverter circuit, as disclosed in a co- 
pending application Serial No. 282,176, filed December 9, 
30 1988, entitled Svibthreshold CMOS Amplifier With Offset 
Adaptation, which application is expressly incorporated 
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herein by reference, is cascaded with a sample/hold 
circuit:. In a preferred embodiment such a sample/hold 
circuit includes a capacitor connected to an input through 
* a pass gate. The adaptable CMOS inverter may be adapted 
5 to a selected voltage placed on the capacitor of the 
sample/hold circuit. The voltage on the input of the 
inverter is set by the sample/hold circuit, and the 
circuit is then adapted by illuminating the aperture 
located over the. feedback capacitor in the amplifier with 

10 ultraviolet (U.V.) light as disclosed in co-pending 
application Serial No. 282,176. After the circuit has 
been adapted, the current flowing through it will be 
maximized when the voltage at the input of the circuit 
equals the voltage to which the circuit has been adapted. 

15 In a second aspect of the present invention, a 

plurality of adaptable CMOS amplifiers and associated 
sample/hold circuits are placed in a m by n array. The 
sample/hold inputs of all sample/hold amplifiers in one 
column are commonly connected to one of a plurality of 

20 write lines. The write lines are activated by a write 
decoder. The voltage input lines to each row of 
amplifiers in the matrix are commonly connected to a 
voltage input line. The current supplied to all 
amplifiers in a column is commonly provided by a sense 

25 line. 

In order to adapt the matrix of the present 
invention, the voltages to which a given column of the 
matrix is to be adapted are placed onto the input voltage 
lines and into the sample/hold circuits of that column by 

30 activating the appropriate write decoder. The input 
voltages to successive columns are entered sequentially 
into their sample/hold circuits. The sequential update of 
the sample/hold voltages is continued, and the entire 
array is exposed to ultraviolet light for a period of time 

35 long enough to adapt the circuits. 
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Brief Description Of Th e Drawings 

FIG. la is a schematic diagram of an adaptive CMOS 
inverting amplifier circuit according to a preferred 
embodiment of the present invention. 
5 PIG* lb is a plot of output current as a function of 

input voltage for the circuit of FIG. la. 

FIG. 2 is a circuit diagram of a matrix containing a 
plurality of the circuits in FIG. 1. 

FIG. 3a is a schematic diagram of one embodiment of 
10 a combination of a sample/hold circuit with a CMOS 
adaptive subthreshold amplifier circuit. 

FIG. 3b is a schematic diagram of a presently- 
preferred embodiment of a sample/hold amplifier for use 
with the present invention. 
15 FIG. 4 is a schematic diagram of a matrix of circuits 

similar to those shown in FIG. 3, but using only a single 
transistor for the sample/hold circuit. 

PIG. 5 is a block diagram of the matrix of FIG. 4 and 
the preprocessing and postprocessing circuits associated 
20 therewith. 

FIG. 6 is a schematic diagram of a sense amplifier 
for use in the postprocessing circuits of the present 
invention. 

FIG. 7 is a schematic diagram of a "winner-taJce-all" 
25 circuit for use with the postprocessing circuits of the 
present invention. 

FIG. 8a is a schematic diagram of an alternative 
embodiment of a CMOS inverting amplifier and error current 
generating circuit for use with the present invention. 
30 FIG. 8b is a graph showing the output current of the 

circuit of FIG. 8a. 

FIG. 9 is a "loser-take-all" circuit for use with the 
postprocessing circuits of FIG. 8a. 



35 



Detailed Descript ion Of A Preferred Embodiment 

Referring first to FIG. la, an adaptive CMOS in- 
verting amplifier 10 includes an N-channel MOS transistor 
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12 having its source connected to a source of negative 
voltage and its drain connected to the drain of a 
P-channel MOS transistor 14. The gates of N-channel MOS 
transistor 12 and P-channel MOS transistor 14 are con- 
5 nected together to an input node 16. The source of 

P^channel MOS transistor 14 is connected to a source of 
sense voltage V^^^^. A first capacitor is connected 
between the input voltage source Vj and input node 16. A 
second capacitor is connected between input node 16 and 

10 the output of inverting amplifier 10, the common con- 
nection of the drain of N-channel MOS transistor 12 and 
the drain of P-channel MOS transistor 14. An area above 
in the integrated circuit die containing the circuit of 
FIG. 1 is transparent to U.V. light. The theory and 

15 operation of amplifier 10 is fully disclosed in co-pending 
application Serial No. 282,176, expressly incorporated 
herein by reference. 

The amplifiers disclosed in this specification, and 
the amplifiers disclosed in co-pending application serial 

20 no. 282,176 shall, for the purposes of the specification 
and claims herein, be referred to as "adaptive" 
cunplifiers. 

Referring now to FIG. lb, curves 20, 22, and 24 show 
the current drawn by the inverting CMOS amplifier 10 as a 

25 function of input voltage for three cases in which the 
circuit has been adapted to different voltages VI, V2, and 
V3 respectively. A characteristic of the CMOS inverting 
amplifier 10 of FIG. la is that when the input voltage is 
equal to the voltage to which the circuit has been 

30 adapted, the output current is maximized as shown by 
curves 20, 22, and 24. 

Referring now to FIG. 2, a matrix of four CMOS 
inverting amplifiers is shown. The first amplifier 10a 
and second amplifier 10b have their inputs connected 

35 together to a input voltage- line V^. Amplifiers 10c and 
lOd have their inputs connected together to a second input 
voltage line Vg. The sources of the P-channel transistors 
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. in amplifiers lOa and lOc are connected to a common sense 
line 34. The source of the P-channel MOS transistors in 
amplifiers 10b and lOd are connected to sense line 36. 

One drawback of the array of FIG. 2 is that each 
5 amplifier on a common input voltage line is constrained to 
be adapted to the same voltage since the problem of 
selectively exposing the feedback capacitors of different 
amplifiers to ultraviolet light is significant. 

One solution to this dilemma is the embodiment shown 

10 in FIG. 3a. Referring now to FIG. 3a, subthreshold CMOS 
inverting amplifier 10 has its input driven by a sample 
hold circuit comprising a pass gate including an N-channel 
MOS transistor 40 and P-channel MOS transistor 42. 
Complementary signals SEL and SEL are used to control the 

15 pass gate. When the pass gate is activated, the voltage 
appearing at the input of the pass gate is stored on 
capacitor 44. Because different voltages may be stored on 
the individual capacitors in the samjple/hold circuits, 
individual amplifiers in the same row in an array may be 

20 adapted to different voltages. 

Referring now to FIG. 3b, a presently-preferred 
embodiment of a sample/hold circuit for use with the 
present invention is disclosed. A first P-channel 
transistor channel 46 has its source connected to a source 

25 of positive voltage Vdd, shown at reference numeral 48, 
and its drain connected to the drain of a first N-channel 
MOS transistor 50. The gate of first P-channel MOS 
transistor 46 is connected to the drain of first N-channel 
MOS transistor 50. The source of first N-channel MOS 

30 transistor 50 is connected to the drain of second N- 
channel MOS transistor 52. The source of first N-channel. 
transistor 52 is connected to a source of negative voltage 
Vss, shown at reference numeral 54. The gate of first N- 
channel MOS transistor 50 is connected to an input node 

35 V. . 

A second P-channel MOS transistor 56 has its source 
connected to the source of positive voltage 48, its drain 
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connected to the drain of third N-channel MOS transistor 
58, and its gate connected to the gate of first P-channel 
MOS transistor 46 • The source of third N-channel MOS 
transistor 58 is connected to the source of first N- 
5 channel MOS transistor 50 and the drain of second N- 
channel MOS transistor 52. The gate of third N-channel 
MOS transistor 58 is connected to its drain, and to one 
plate of a holding capacitor 44. The other plate of 
holding capacitor 44 is connected to a source of fixed 
10 voltage, preferably Vss, shown at reference numeral 54. 
The output node 60 of the sample/hold circuit of FIG. 3b, 
is the common connection of the drain and gate of third N- 
channel MOS transistor 58 and the first plate of hold 
capacitor 44. 

15 The circuit of FIG. 3b, Jcnown to those of ordinary 

skill in the art as a 5 transistor transconductance 
amplifier, is superior to the circuit of FIG. 3a because 
the output Vjjjjjj, is not disturbed when the select input, the 
gate of second N-chaniiel MOS transistor 52, goes low, 

20 because any transient current is divided equally between 
first N-channel MOS transistor 50 and third N-channel MOS 
transistor 58. Therefore the effect on Vj,^^^ cancels to the 
first order. 

Referring now to FIG. 4, an illustrative embodiment 
25 of an array of individual adaptable CMOS inverting 
amplifiers is shown. A pass gate consisting of a single 
N-channel MOS transistor 62 and a capacitor 44 form the 
sample/hold circuit in each of the CMOS inverting 
amplifiers of the array of FIG. 4. 
30 A write decoder . 64 has a plurality of output lines 

which drive the select lines of the sample/hold circuits. 
Two such lines, 66 and 68 are shown driving the first and 
second columns in the array, respectively. Write decoder 
64 is a high active one of n decoder. Such decoders are 
35 well known in the art. In a presently-preferred embodi- 
ment, write decoder 64 will be integrated with the matrix 
and a CMOS decoder is preferred. 
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Input voltage lines 70 and 72 are connected to all of 
the amplifiers in the first and second rows of the array, 
respectively. Current sense lines 74 and 76 are connected 
to the sources of the P~channel transistors in the first 
5 and second columns of the array respectively. 

The array of FIG. 4 may be programmed one column at 
a time by addressing a column of the array by placing a 
positive voltage on the select line (shown in FIG. 4 as 
either reference numeral 66 or 68) and placing the input 

10 voltage to which the individual amplifiers in the 
addressed column are to be adapted on each voltage input 
line (shown in FIG. 4 as reference numerals 70 and 72). 
After all of the input voltages to which all amplifiers 
are to be adapted have been placed in the sample/hold 

15 circuits the entire array is exposed to a source of 
ultraviolet light. 

The circuit may then be used to associate a series of 
input vectors with the series of vectors stored as the 
adapted input voltages. All select lines are simul- 

20 taneously brought to a positive level and the voltage 
vector to be associated is placed on the voltage input 
lines. The input vector will be closest to the vector 
stored in one particular column. The input will .cause the 
current in the current sense line associated with that 

25 column to be the highest value. 

Referring now to FIG. 5, a matrix M.j of elements 
similar to those shown in FIG. 3a, shown at reference 
numeral 78, is connected to a plurality of inputs through 
a preprocessing circuit 80 and to a set of outputs through 

30 a post processing circuit 82. 

Preprocessing circuit 80 may be used for the purpose 
of normalizing the inputs to the array. in real-world 
applications these inputs may have peak voltage values 
which vary over a wide range of voltages. The 

35 preprocessing circuit 80 may be used to normalize those 
voltages so that the magnitude of the resultant voltage 
vector is constant. An example of such a circuit is 
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disclosed in B. Gilbert/ A Monolithic 16~channel Analog 
Array Normal izer, IEEE Journal of Solid State Circuits, 
Vol* SC-19, No. 6, p. 956, December 1984, which is 
expressly Incorporated herein by reference* 
5 The sense lines associated with each of the columns 

in the array of FIG. 4 may each be connected to a sense 
amplifier circuit such as the one shown in FIG. 6. In the 
circuit of FIG. 6, a sense amplifier 84 includes an 
operational amplifier 86 having its noninverting input 

10 connected to V^^, and its inverting input connected to 
the sense line of a particular column. An N-channel MOS 
transistor 88 has its drain connected to V^^, a source of 
positive voltage, its source connected to the inverting 
input of operational amplifier 86 and its gate connected 

15 to the output of operational amplifier 86. A second N- 
channel HOS transistor 90 has its gate connected to the 
output of operational amplifier 8.6 and its source 
connected to V^g^^g. 

The current flowing into the drain of N-channel MOS 

20 transistor 90, shown in FIG. 6 as I'^ will be related to 
the current flowing in the sense line of the array of FIG. 
4 with which the circuit of FIG. 6 is associated. The 
circuit of FIG, 6 holds the sense line at the voltage V^^^^ 
while making a replica of the current I*^ for use in the 

25 post processing circuits. 

Referring now to FIG. 7, a component of the post 
processing circuits 82 of FIG. 5, known as a "winner-take- 
all" circuit is shown* This circuit is completely dis- 
closed in co-pending application Serial No. 277,795, filed 

30 November 30, 1988, which is expressly incorporated herein 
by reference. 

In FIG. 7, winner- take-all circuit 92 includes a 
plurality of sections, two of which are shown. In the 
first section, a current mirror consisting of P-channel 
35 current mirror MOS transistor 94 and P-channel MOS current 
mirror transistor 96. The drain of P-channel current 
mirror transistor 96 is connected to the drain of N- 
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channel MOS transistor 98. The source of N-channel MOS 
transistor 98 is connected to a source of negative 
voltage, shown in FIG. 7 as V^^ or ground. 

Another N-channel MOS transistor 100 has its source 
5 connected to a common gate line 102, its drain connected 
to a source of positive voltage and its gate connected to 
the common connection of the drain of N-channel MOS 
transistor 98 and the drain of P-channel MOS current 
mirror transistor 96. The node to which the gate of N- 

10 channel MOS transistor lOO is connected is the output 
current node for the column of the array of FIG, 6 
associated with that section, shown in FIG. 7 as OUT^. 

The second section of the circuit for the second 
column includes a current mirror comprising P-channel MOS 

15 current mirror transistors 104 and 106 and N-channel MOS 
transistors 108 and 110, connected in exactly the same 
manner as are the transistors for the first section. 
Common gate line 102 is connected to the drain of N- 
channel MOS transistor 112 whose purpose is to place a 

20 bias on gate line 102. 

While the operation of the winner take all circuit in 
FIG. 7 is fully described in co-pending application Serial 
No. 277,795 incorporated herein by reference, briefly, 
follower transistors 100, 110, etc., pull the common gate 

25 line 102 up to a voltage at which the current through the 
common gate pulldown devices 98, 108, etc., is equal to 
the maximum input current. This turns off all sections in 
the circuit except for the one having the largest current 
flowing through it. 

30 An alternate embodiment of the synaptic element of 

FIG. 3a is shown in FIGS. 8a and 8b. 

Referring now to FIG. 8a, a CMOS inverting amplifier 
10 is driven by a sample/hold circuit including the pass 
gate comprising N-channel transistor 40 and P-channel 

35 transistor 42 driven by SEL and SEL, charging capacitor 
44. In the circuit of FIG. 8a, the source of the P- 
channel transistor in amplifier 10 is connected to a 
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source of reference voltage V^.^^ and its output is taken 
from the common connection between the drains of the P- 
channel and the N-channel transistors. * 
However, unlike the embodiment of FIGS. 3a and 3b, 
5 the embodiment of FIG. 8a drives two follower transistors, 
N-channel transistor 114 and P-channel transistor 116. N- 
channel transistor 114 is biased by an error voltage +Eru 
and P-channel transistor 116 is biased by an error voltage 
-E)j^. The error lines, reference numerals 118 and 120 
10 respectively, are biased such that if the output voltage 
of the cimplifier 10 moves away from the center point, the 
current drawn starts to increase quadrat ically. 

The total output current of the circuit of FIG. 8a is 
shown in PIG. 8b. The width of the curve in FIG. 8b is 
15 determined by and by the choice of the + and - error 
voltages which are used to bias lines 118 and 120. In 
both this circuit and that of FIG. 3a, the error current 
is a monotonic function of the unsigned, difference between 
the amplifier output voltage and the output voltage at 
20 which the amplifier was adapted. 

Referring now to FIG. 9, a post processing circuit 
for use with an array of circuit such as that shown in 
FIG. 8a is disclosed. This circuit may be referred to as 
a "loser-take-all circuit" 122 which causes the column 
25 having the lowest output current to be passed to the 
output. One section of a plurality of sections is shown. 

In FIG. 9, a current mirror composed of P-channel MOS 
current mirror transistors 124 and 126 has its first P- 
channel MOS current mirror transistor 124 connected to the 
30 I'coix node and has its second P-channel MOS transistor 
connect in series with an N-channel MOS transistor 128. 
The N-chaiinel MOS transistor 128 has its source connected 
to a source of negative voltage, shown in FIG. 9 as ground 
or Vss, and its gate connected to a coimuon gate line 130 
35 for N-channel transistors for all other legs driving all 

the other columns from the matrix. A third P-channel ' 
transistor 132 has its gate connected to the common 
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connection of the drains of N-channel transistor 128 and 
P-channel transistor 126, its drain connected to ground, 
and its source connected to the cominon gate line 130 of 
the post processing circuit. A bias transistor 134 
5 supplies a bias to common gate line 130. 

The loser-take-all circuit of FIG* 9 operates as 
follows. A bias current is injected into the common gate 
line 130 by bias transistor 134. That current is just 
balanced by the current out of the node V^^^ passing through 
10 N-channel MOS transistor 128 of the section whose V is 

out 

the least. The 'lowest V^^ will correspond to the smallest 
input current in the current mirror, shown in FIG. 9 as 
124 and 126, in the section having that least current. In 
operation, the voltage on node 130 will rise until the N- 

15 channel MOS transistor, shown associated with the first 
section as transistor 128, of one section is able to pull 
^aat ^ow. This single low output will drain the bias 
current from node 130, and its voltage will stabilize such 
that the drain current of N-channel . transistor 128 just 

20 balances in the input current. All sections with higher 
input currents will have V^^^ high. Hence, this circuit 
functions as an encoder of the stage with the minimum 
input current. 

When used with an array of cells with the type shown 

25 in FIG. 8a, the circuit of FIG. 9* encodes the column which 
has the minimum error current, and therefore whose stored 
vector is closest to the input vector. 

While a presently-preferred embodiment of the 
invention has been disclosed, those of ordinary skill in 

30 the art will, from an examination of the within disclosure 
and drawings be able to configure other embodiments of the. 
invention. These other embodiments are intended to fall 
within the scope of the present invention which is to be 
limited only by the scope of the appended claims. 
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Claims 

1« An electronic circuit including in combination: 
an adaptive amplifier having an input node and an 

output node, said adaptive amplifier including means for 
5 adaptation by ultraviolet light to a preselected voltage 

placed upon said input node, 

a sample/hold circuit having an input, a select 

input, and an output, said output connected to the input 

node of said adaptive amplifier. 

10 2« The electronic circuit of claim 1 wherein the 

sample/hold circuit is a five- transistor transconductance 
amplifier having a capacitor connected between said output 
and a source of fixed voltage. 

3. The electronic circuit of claim 1 wherein said 
15 sample/hold circuit comprises a pass-transistor having a 

gate, a source and a drain, and a capacitor, said source 
of said pass-transistor being said input of said 
sample/hold circuit, said drain of said pass- transistor 
being said output of said sample/hold circuit, said 
20 capacitor connected between said output and a source of 
fixed voltage, and said gate of said pass-transistor being 
the select input of said sample/hold circuit. 

4. An electronic circuit including in combination: 
an adaptive amplifier having an input node and 

25 an output node, said adaptive amplifier including means 

for adaptation by ultraviolet light to a preselected 

voltage placed upon said input node, 

sample/hold circuit means, connected to the 

input node of said adaptive amplifier, for selectively 
30 storing an analog voltage at the input node of said 

adaptive amplifier. 

5. An electronic circuit including in combination: 
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an adaptive amplifier having an input node and 
an output node, said adaptive amplifier including means 
for adaptation by ultraviolet light to a preselected 
voltage placed upon said input node, 
5 a sample/hold circuit having an input, a select 

input, and an output, said output connected to the input 
node of said adaptive amplifier, and 

means within said adaptive amplifier for 
generating an error current which changes monotonically as 
10 a function of the voltage difference between the actual 
voltage on said output node of said adaptive amplifier and 
said preselected voltage. 

6. The electronic circuit of claim 5 wherein said 
adaptive amplifier is a CMOS inverter, and said error 

15 current is the current drawn by said inverter. 

7. The electronic circuit cf claim 6 wherein the 
sample/hold circuit is a five-transistor transconductance 
amplifier having a capacitor connected between said output 
and a source of fixed voltage* 

20 8. The electronic circuit of claim 6 wherein said 

sample/hold circuit comprises a pass-transistor having a 
gate, a source and a drain, and a capacitor, said source 
of said pass-transistor being the input of said 
sample/hold circuit, said capacitor connected between said 

25 output and a source of fixed voltage, said drain of said 
pass- transistor being the output of said sample/hold 
circuit, and said gate of said pass-transistor being the 
select input of said sample/hold circuit. 

9. An array of electronic circuits arranged in rows 
30 and columns, each of said electronic circuits including an 
adaptive amplifier having an input node and an output node 
and means for generating an error current, and a 
sample/hold circuit having an input, an output, and a 
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select input, the output of said sample/hold circuit 
connected to the input node of said adaptive amplifier, 
the select inputs of all sample/hold circuits associated 
with the electronic circuits in a given column of said 
5 array connected in common to one of a plurality of write 
lines, and the error current generated from each adaptive 
amplifier in a column connected in common to one of a 
plurality of output sense lines, and the inputs of all the 
sample/hold circuits in a row of said array connected in 
10 common to one of a plurality of input voltage lines. 

10. The array of claim 9 wherein said means for 
generating an error current generates an error current 
decreasing monotonically with error and further including: 

write decoder means connected to said plurality of 
15 write lines for selectively activating one of said write 
lines at a time, 

a plurality of sense amplifiers, one of said sense 
amplifiers having its input connected to each of said 
sense lines, and 
20 means coxmected to the outputs of said sense 

amplifiers for indicating which one of said plurality of 
output sense lines is drawing more current than the other 
ones of said plurality of output sense lines. 

11. The array of claim 9 wherein said means for 
25 generating an error current generates an error current 

increasing monotonically with error and further including: 
write decoder means connected to said plurality of 
write lines for selectively activating one of said write 
lines at a time, 
30 a plurality of sense amplifiers, one of said sense 

amplifiers having its input connected to each of said 
sense lines, and 
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means connected to the outputs of said sense 
amplifiers for sensing which one of said plurality of 
output sense lines is drawing less current than the other 
ones of said plurality of output sense lines. 
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